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(54) AV information processing apparatus and computer program 



(57) An AV information processing apparatus (A) is 
provided with: an AV information accumulating device 
(14) for accumulating AV information including at least 
one of audio information, video information and data in- 
formation corresponding to at least one of the audio in- 
formation and the video information; a requesting device 
(7) for requesting an execution of an information proc- 
ess, which uses the accumulated AV information, from 
an external portion; an executing device (10) for execut- 
ing the requested information process by using the ac- 



cumulated AV information; a related information accu- 
mulating device (15) for accumulating related informa- 
tion, which is related to the accumulated AV information; 
and an outputting device (5) for searching the related 
information accumulating device for the related informa- 
tion corresponding to a non-executable process, which 
is not executable by the executing device, and output- 
ting the searched related information to an external por- 
tion when the non-executable process is requested by 
the requesting device. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001] The present invention relates to an AV (Audio 
Visual) information processing apparatus for processing 
AV information including at least one of (i) audio infor- 
mation having voice information and music information, 
(ii) video information having dynamic picture information 
and static picture information and (iii) data information, 
such as program data, character data or the like, corre- 
sponding to at least one of the audio information and the 
video information (hereafter, which are merely referred 
to as "AV information"). More particularly, the present 
invention relates to an AV information processing appa- 
ratus for carrying out a process corresponding to a proc- 
ess request from a user. 

[0002] The present invention also relates to a compu- 
ter program for processing the AV inf ormation , which en- 
ables a computer to carry out such a process corre- 
sponding to the process request from the user. 

2. Description of the Related Art 

[0003] For example, in case that user's desired AV in- 
formation is reproduced and displayed by the AV infor- 
mation processing apparatus containing a plurality of in- 
formation record media, such as hard discs and the like, 
on which a plurality of AV information are recorded in 
advance, the user owns in advance "reproduction-relat- 
ed-inf ormation" (e.g., identification information to spec- 
ify the information record medium on which a song name 
or the AV information desired to be viewed is recorded) 
with regard to the AV information that the user desires 
to view, and inputs this owned reproduction-related-in- 
formation to the AV information processing apparatus 
so as to specify the AV information to be viewed. Then, 
the user instructs the AV information processing appa- 
ratus to start the reproduction of the specified AV infor- 
mation, and accordingly views the desired AV informa- 
tion. 

[0004] On the other hand, in case that user's desired 
AV information is recorded onto any one of the informa- 
tion record media, the user owns in advance "record- 
related-information" (e.g., specifying information to 
specify the information record medium, or record posi- 
tion information to specify a record position on the spec- 
ified information record medium) with regard to the in- 
formation record medium onto which the AV information 
is desired to be recorded, and inputs this owned record- 
related-information to the AV information processing ap- 
paratus so as to specify the information record medium 
onto which the AV information is desired to be recorded. 
Then, the user instructs the AV information processing 
apparatus to start recording, and accordingly records 
the desired AV information. 



[0005] However, in the above-mentioned reproduc- 
tion of the AV information, if a record amount of the AV 
information is huge and if the number and the kind of 
the information record media for recording the AV infor- 

5 mation are various (for example, such as a CD (Com- 
pact Disc), a DVD, a cassette tape, a MD (Mini Disc) 
and the like), a procedure of searching for the AV infor- 
mation by using the reproduction-related-information 
becomes complex, which results in a problem that the 

10 user who is not accustomed to the treatment of the AV 
information processing apparatus has difficulty in treat- 
ing it. 

[0006] Moreover, even in the case of recording the AV 
information, if the kind and the number of information 

is record media for recording are various, a procedure 
necessary for recording becomes complex, which re- 
sults in the problem that the user who is not accustomed 
to the treatment of the AV information processing appa- 
ratus has the difficulty in treating it, similarly to the case 

20 of the reproduction. 

SUMMARY OF THE INVENTION 

[0007] The present invention is proposed in view of 
25 the above mentioned problems. It is therefore an object 
of the present invention to provide an AV information 
processing apparatus, which allows a user to carry out 
a necessary procedure of reproducing and/or recording 
many kinds of AV information and a large quantity of AV 
30 information easily and quickly, in other words, which al- 
lows the user to carry out the necessary procedure in a 
so-called user-friendly manner, even if the user who is 
not accustomed to a treatment of the AV information 
processing apparatus carries out the necessary proce- 
35 dure, as well as a computer program, which enables a 
computer to function as the AV information processing 
apparatus. 

[0008] The above object of the present invention can 
be also achieved by an AV information processing ap- 

^o paratus provided with: an AV information accumulating 
device, such as an AV information recording portion or 
the like, for accumulating AV information including at 
least one of audio information, video information and da- 
ta information corresponding to at least one of the audio 

45 information and the video information; a requesting de- 
vice, such as a microphone or the like, for requesting an 
execution of an information process, which uses the ac- 
cumulated AV information, from an external portion; an 
executing device, such as an AV control agent or the 

so like, for executing th e req uested information process by 
using the accumulated AV information; a related infor- 
mation accumulating device, such as an AV information 
database or the like, for accumulating related informa- 
tion, which is related to the accumulated AV information; 

55 and an outputting device, such as an interactive agent 
or the like, for searching the related information accu- 
mulating device for the related information correspond- 
ing to a non-executable process, which is not executa- 
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ble by the executing device, and outputting the searched 
related information to an external portion when the non- 
executable process is requested by the requesting de- 
vice. 

[0009] According to the AV information processing 
apparatus of the present invention, the information proc- 
ess is executed when the information process, which is 
executable by using the accumulated AV information, is 
requested. On the other hand, even if the information 
process, which is not executable by using the AV infor- 
mation, is requested, the corresponding related infor- 
mation is searched and outputted. Accordingly, it is pos- 
sible for the user to obtain some process result when 
the user requests the information process to the AV in- 
formation processing apparatus. Thus, even the user 
who is not accustomed to the treatment of the AV infor- 
mation processing apparatus can easily treat the AV in- 
formation processing apparatus without dissatisfaction. 
In this way, it is possible for the AV information process- 
ing apparatus to execute the necessary information 
process in the user friendly manner. 
[0010] In one aspect of the AV information processing 
apparatus of the present invention, the outputting device 
searches the related information accumulating device 
for a plurality of the related information, whose response 
manners to the non-executable process are different 
from each other, and outputs the searched plurality of 
the related information to the external portion. 
[0011] According to this aspect, since the plurality of 
related information, whose response manners are dif- 
ferent from each other, are outputted, it is possible for 
the AV information processing apparatus to execute the 
information process in the user friendly manner. 
[0012] In another aspect of the AV information 
processing apparatus of the present invention, the out- 
putting device outputs the searched related information 
corresponding to the non-executable process to the ex- 
ternal portion by using at least one of a voice and a pic- 
ture. 

[0013] According to this aspect, since the related in- 
formation is outputted by using the voice and/or picture, 
it is possible to output the related information in such a 
manner that the user can easily recognize it. 
[0014] In another aspect of the AV information 
processing apparatus of the present invention, the ap- 
paratus is further provided with a sub-outputting device, 
such as an interactive agent or the like, for outputting 
information indicating a fact that no process can be ex- 
ecuted by the AV information processing apparatus if an 
impossible process, which is impossible to execute 
even by using the related information, is requested by 
the requesting device. 

[0015] According to this aspect, if the information 
process, which is not executable by the AV information 
processing apparatus, is requested, the fact that no 
process can be executed is outputted. Thus, it is possi- 
ble for the user to obtain some process result when the 
user requests the information process to the AV infor- 



mation processing apparatus. In this way, it is possible 
for the AV information processing apparatus to execute 
the necessary information process in a further user 
friendly manner. 
s [0016] In another aspect of the AV information 
processing apparatus of the present invention, the re- 
questing device is provided with a voice receiving de- 
vice, such as a microphone or the like, for receiving a 
request of the information process from the external por- 
tion through a voice. 

[0017] According to this aspect, it is possible for the 
user to easily request the information process by using 
the voice. 

[001 8] The above object of the present invention can 
be also achieved by a computer program of instructions 
executable by a computer to perform steps to execute 
information process requested by a user, in an AV infor- 
mation processing apparatus provided with: an AV in- 
formation accumulating device for accumulating AV in- 
formation including at least one of audio information, 
video information and data information corresponding 
to at least one of the audio information and the video 
information; and a related information accumulating de- 
vice for accumulating related information, which is relat- 
ed to the accumulated AV information. The steps in- 
clude: a requesting step of requesting an execution of 
an information process, which uses the accumulated AV 
information, from an external portion; an executing step 
of executing the requested information process by using 
the accumulated AV information; and an outputting step 
of searching the related information accumulating de- 
vice for the related information corresponding to a non- 
executable process, which is not executable by the ex- 
ecuting step, and outputting the searched related infor- 
mation to an external portion when the non-executable 
process is requested by the requesting step. 
[0019] According to the computer program of the 
present invention, as the computer reads in and exe- 
cutes the computer program, it is possible to realize the 
above described AV information processing apparatus 
of the present invention on the computer. 
[0020] In one aspect of the computer program of the 
present invention, the outputting step searches the re- 
lated information accumulating device for a plurality of 
the related information, whose response manners to the 
non-executable process are different from each other, 
and outputs the searched plurality of the related infor- 
mation to the external portion. 
[0021 ] According to this aspect, since the plurality of 
related information, whose response manners are dif- 
ferent from each other, are outputted, it is possible for 
the AV information processing apparatus to execute the 
information process in the user friendly manner. 
[0022] In another aspect of the computer program of 
the present invention, the outputting step outputs the 
searched related information corresponding to the non- 
executable process to the external portion by using at 
least one of a voice and a picture. 
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[0023] According to this aspect, since the related in- 
formation is outputted by using the voice and/or picture, 
it is possible to output the related information in such a 
manner that the user can easily recognize it. 
[0024] In another aspect of the computer program of 
the present invention, the steps further include a sub- 
outputting step of outputting information indicating a fact 
that no process can be executed by the AV information 
processing apparatus if an impossible process, which is 
impossible to execute even by using the related infor- 
mation, is requested by the requesting step. 
[0025] According to this aspect, if the information 
process, which is not executable by the AV information 
processing apparatus, is requested, the fact that no 
process can be executed is outputted. Thus, it is possi- 
ble for the user to obtain some process result when the 
user requests the information process to the AV infor- 
mation processing apparatus. In this way, it is possible 
for the AV information processing apparatus to execute 
the necessary information process in a further user 
friendly manner. 

[0026] In another aspect of the computer program of 
the present invention, the requesting step is provided 
with a voice receiving step of receiving a request of the 
information process from the external portion through a 
voice. 

[0027] According to this aspect, it is possible for the 
user to easily request the information process by using 
the voice. 

[0028] The nature, utility, and further features of this 
invention will be more clearly apparent from the follow- 
ing detailed description with respect to preferred em- 
bodiments of the invention when read in conjunction 
with the accompanying drawings briefly described be- 
low. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0029] FIG.1 is a block diagram showing a schematic 
configuration of an AV information processing appara- 
tus as an embodiment of the present invention; 
[0030] FIG. 2 is a flowchart showing respective proc- 
esses constituting an AV information process schemat- 
ically in a module manner in the embodiment; 
[0031 ] FIG.3 is a flowchart showing detailed portions 
of a login process in the embodiment; 
[0032] FIG.4 is a flowchart showing detailed portions 
of an input process in the embodiment; 
[CC33] FIG.5 is a flowchart showing detailed portions 
of a request analysis process in the embodiment; 
[0034] FIG. 6 is a flowchart showing detailed portions 
of an accumulated information process in the embodi- 
ment; 

[0035] FIG. 7 is a flowchart showing detailed portions 
of a user response process in the embodiment; 
[0036] FIG. 8 is a flowchart showing detailed portions 
of an output process in the embodiment; 
[0037] FIG. 9 is a flowchart showing detailed portions 



of a reproducing process in the embodiment; 
[0038] FIG.1 0 is a flowchart showing detailed portions 
of a recording process in the embodiment; 
[0039] FIG.1 1 is a flowchart showing detailed portions 
5 of a logout process in the embodiment; and 

[0040] FIG.1 2 is a flowchart showing detailed portions 
of an information download process in the embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED 
10 EMBODIMENTS 

[0041] Preferred embodiments of the present inven- 
tion will be explained below with reference to the draw- 
ings. 

is [0042] In the following embodiment, the present in- 
vention is applied to an AV information processing ap- 
paratus, which can reproduce one AV information from 
an AV information recording portion, such as a hard disc 
or the like, on which the AV information is recorded, and 
also can record another AV information onto an informa- 
tion record medium by specifying it. 

( I ) AV Information processing apparatus 

[0043] At first, a schematic configuration of the AV in- 
formation processing apparatus as the embodiment of 
the present invention is described with reference to FIG. 
1 . FIG.1 is a block diagram showing the schematic con- 
figuration of the AV information processing apparatus. 
[0044] In FIG.1 , an AV information processing appa- 
ratus A as the embodiment is installed within a single 
house. The AV information processing apparatus A is 
provided with: an AV information accumulating appara- 
tus S; an audio memory recorder 19 which can record 
and/or reproduce AV information onto or from an audio 
memory, such as a semiconductor (solid) memory or a 
recordable optical disc (e.g., CD-R (CD-Recordable), 
DVD-R (DVD-recordable), DVD-RAM (DVD-Random 
Access Memory)) and the like; a cassette deck 21 ; a CD 
player 23; a DVD player 25; and an MD player/recorder 
27. These audio memory recorder 19 etc., and the AV 
information accumulating apparatus S are connected to 
each other through a network N such as a home LAN 
(Local Area Network) so as to send and receive infor- 
mation to and from each other. 
[0045] On the other hand, the AV information accu- 
mulating apparatus S is provided with: a voice recognize 
agent 2, to which a microphone 1 serving as one exam- 
ple of a requesting device and one example of a voice 
receiving device is connected; a language analyze & 
compose agent 3; a user learn agent 4; an interactive 
agent 5; an edit agent 6; a voice synthesize agent 8 to 
which a speaker 7 is connected; a system manage 
agent 9; an AV control agent 1 0 serving as one example 
of an executing device and also one example of a com- 
pleting device; a search agent 11 ; a database agent 12; 
a download agent 13; a display 18 containing a system 
manage agent 17; an AV information recording portion 
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14 serving as one example of an AV information accu- 
mulating device which is actually constituted by a hard 
disc and Its driver; and an AV information database 15 
as one example of the AV information accumulating de- 
vice and one example of a related information accumu- 
lating device. These respective agents, the AV informa- 
tion recording portion 14 and the information database 

15 are connected through a communication bus B so as 
to send and receive information to and from each other. 
[0046] Also, the download agent 13 is connected to 
an external network 16, for example, such as the Inter- 
net and the like, so as to send and receive information 
to and from it. 

[0047] On the other hand, the audio memory recorder 
19, the cassette deck 21 , the CD player 23, the DVD 
player 25 and the MD player/recorder 27 include system 
manage agents 20, 22, 24, 26 and 28 respectively which 
are connected to the network N and control the opera- 
tions of the respective devices. 
[0048] At this time, the respective system manage 
agents 1 7, 20, 22, 24, 26 and 28 are respectively con- 
nected to the system manage agent 9 within the AV in- 
formation accumulating apparatus S through the net- 
work N so as to send and receive information to and 
from the system manage agent 9. 
[0049] Here, the "agent" is a module (i.e., a program 
module) having functions of self-controlling, cooperat- 
ing and learning for each agent to judge what is to be 
processed and which output is to be outputted in accord- 
ance with a requested content and then perform the 
judged process and output the judged output, so that 
the process based on the requested content can be pos- 
itively carried out under an independent judgment 
standard. At this time, each agent is realized by means 
of a CPU and the like as a computer for carrying out the 
process on the basis of a program corresponding to a 
function of each agent. 

[0050] Each of the agents respectively carries out the 
independently given process. Thus, for example, if any 
one of the agents cannot be operated because of any 
reason, the other agents can continue to carry out the 
processes as they are, except the process related to this 
disabled agent. 

[0051] By the way, the "agent" is described in detail, 
for example, in "Agent Orientation From Object Orien- 
tation", written by Shinichi Motoida and Akihiko Osuga, 
Soft Bank Corporation, in May 1 998, and the like. 
[0052] Next, the schematic operations of the respec- 
tive sections will be described below. 
[0053] At first, under a control of the system manage 
agent 20, the audio memory recorder 1 9 records AV in- 
formation to be recorded, which is outputted through the 
network N from the AV information accumulating appa- 
ratus S, to a specified information record medium such 
as the above-mentioned semiconductor memory and 
the like, and also outputs the AV information recorded 
on the audio memory, through the network N to the AV 
information accumulating apparatus S. 



[0054] On the other hand, under a control of the sys- 
tem manage agent 22, the cassette deck 21 records the 
AV information to be recorded, which is outputted 
through the network N from the AV information accumu- 
5 lating apparatus S, to a mounted cassette tape, and also 
outputs the AV information recorded in the cassette tape 
through the network N to the AV information accumulat- 
ing apparatus S. 

[0055] Under a control of the system manage agent 
10 24, the CD player 23 outputs the AV information record- 
ed on a mounted CD, through the network N to the AV 
information accumulating apparatus S. 
[0056] Under a control of the system manage agent 
26, the DVD player 25 outputs the AV information re- 
's corded on a mounted DVD through the network N to the 
AV information accumulating apparatus S. 
[0057] Under a control of the system manage agent 
28, the MD player/recorder 27 records the AV informa- 
tion to be recorded, which is outputted through the net- 
20 work N from the AV information accumulating apparatus 
S, onto a mounted MD, and also outputs the AV infor- 
mation recorded on the MD through the network N to 
the AV information accumulating apparatus S. 
[0058] In linkage with these respective devices con- 
25 nected as mentioned above, the AV information accu- 
mulating apparatus S carries out processes of output- 
ting one AV information through the speaker 7 into a 
house, in response to a request received through a 
voice from a user as detailed below, and also recording 
30 another AV information to either one of the information 
record media. 

[0059] Next, the typical operations in the respective 
agents and the like included in the AV information accu- 
mulating apparatus S will be described below with ref- 
35 erence to FIG.1 . 

[0060] At first, the AV information recording portion 1 4 
accumulates therein a plurality of kinds of AV informa- 
tion under a condition that they can be mutually identi- 
fied and read. 

40 [0061 ] Next, the AV information database 1 5 records 
therein the attribute information indicative of the respec- 
tive AV information recorded in the AV information re- 
cording portion 14 (e.g., identification information to 
identify a name of the recorded AV information, a cate- 

45 gory to which the recorded AV information belongs, a 
time necessary for reproducing the recorded AV infor- 
mation, and an information record medium to which the 
AV information is recorded, or relation information indi- 
cating such a fact that the recorded AV information has 

so been used as a theme song of a movie or the like) under 
a condition that a plurality of the recorded AV information 
can be mutually identified. 

[0062] On the other hand, the voice recognize agent 
2 is the agent which has a function referred to as a "voice 
55 recognition engine" and carries out a process of a rela- 
tively low intelligence level. Actually, it recognizes the 
content of a voice signal corresponding to a voice of a 
user received from the microphone 1 (i.e., a voice indic- 
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ative of a process to be carried out by using the AV in- 
formation processing apparatus A or an answer corre- 
sponding to a voice outputted from the speaker 7 or the 
like), and outputs the content information indicative of 
the recognized content through the bus B to the Ian- 5 
guage analyze & compose agent 3. 
[0063] The language analyze & compose agent 3 is 
the agent for carrying out a process of a high intelligence 
level. Actually, it analyzes the obtained content informa- 
tion, and translates it into a middle language that can be io 
recognized by the other agents except the voice recog- 
nize agent 2 and the voice synthesize agent 8, and then 
outputs the middle language to the bus B. 
[0064] In addition, when obtaining the output informa- 
tion corresponding to the audio information or the re- 15 
sponse voice to be outputted through the speaker 7, 
from the bus B as the middle language, the language 
analyze & compose agent 3 converts it into the audio 
information or the voice signal that can be synthesized 
by the voice synthesize agent 8, and then outputs it to 20 
the voice synthesize agent 8. 

[0065] The voice synthesize agent 8 is the agent 
which has a function referred to as a "voice synthesizing 
engine" and carries out a process of a relatively low in- 
telligence level. Concretely, by using the converted 25 
voice signal or audio information, the voice synthesize 
agent 8 actually synthesizes the voice or audio informa- 
tion for outputting, and then outputs it through the speak- 
er 7 to the user in the house. 

[0066] The interactive agent 5 is the agent for carrying 30 
out a process of a high intelligence level. Concretely, 
through the bus B, the interactive agent 5 carries out a 
process of controlling a relation between the voice rec- 
ognizing process and the voice synthesizing process (e. 
g., a process of controlling a relation between a timing 35 
of the execution of the voice recognizing process in the 
language analyze & compose agent 3 and a timing of 
the execution of the voice synthesizing process, or a 
process of specifying the content of the voice synthesis), 
a process of analyzing and performing the input content *o 
information and the like. 

[0067] The user learn agent 4 is the agent which has 
a so-called "learning function" and carries out a process 
of a high intelligence level. Concretely, when obtaining 
the content information after the voice recognition 45 
through the bus B, the user learn agent 4 carries out the 
processes of dividing it for each user, storing it as a us- 
age history (hereafter, suitably referred to as a "profile"), 
referring to a previous usage history for each user, and 
then analyzing and accumulating a habit and a taste of so 
the user, and also carries out the process of storing a 
request from a user whose process is non-completed 
yet at that point. 

[0068] The edit agent 6 is the agent for carrying out a 
process of an approximately middle intelligence level, ss 
Concretely, it carries out the process of obtaining infor- 
mation through the database agent 12 from the AV in- 
formation database 15, in response to the request from 



the user, and editing a table (list) of reproducible AV in- 
formation, and the like. 

[0069] The search agent 1 1 is the agent for carrying 
out a process of an approximately middle intelligence 
level. Concretely, it carries out the process of searching 
the AV information database 15 through the database 
agent 12 in response to the request from the user, or 
searching the external network 16 through the download 
agent 13. 

[0070] The database agent 1 2 is the agent for carrying 
out a process of a relatively low intelligence level. Con- 
cretely, it carries out the process of, irrespectively of the 
presence or absence of the request from the user, using 
the AV information obtained from the external network 
1 6 through the down load agent 1 3, and updates the con- 
tents of the AV information database 15 and the AV in- 
formation recording portion 14, and also carries out the 
process of the arrangement and the management of the 
information within the AV information database 15 ex- 
cept the searching process, and the like 
[0071 ] The download agent 1 3 is the agent for carry- 
ing out a process of an approximately middle intelli- 
gence level. Concretely, it carries out the process of 
newly obtaining the AV information from the external 
network 16, as the occasion demands, and mainly out- 
putting it to the database agent 12. 
[0072] The AV control agent 1 0 is the agent for carry- 
ing out a process of an approximately middle intelli- 
gence level. Concretely, while mainly sending and re- 
ceiving the information to and from the system manage 
agent 9, the AV control agent 1 0 carries out a reproduc- 
tion control such as a control of a reproduction order of 
AV information, a record control such as a selection of 
an information record medium for recording, and the 
like. 

[0073] Finally, the system manage agent 9 is the 
agent for carrying out a process of a relatively low intel- 
ligence level. Concretely, while sending and receiving 
the information to and from each system manage agent 
connected to the network N and the system manage 
agent 17 within the display 18, the system manage 
agent 9 carries out an interface process and a state 
managing process of each device, such as the audio 
memory recorder 19 connected to the AV information 
accumulating apparatus S and the like. 
[0074] In parallel to those processes, the system 
manage agent 9 jointly carries out a management of a 
signal received from the microphone 1 and a manage- 
ment of a signai outputted to the speaker 7. 

( II ) AV Information Process 

[0075] The AV information process according to the 
present invention to be carried out in the AV inf ormation 
processing apparatus A including the respective agents 
having the above-mentioned configurations will be de- 
scribed below with reference to FIG. 2 to FIG. 12. 
[0076] At first, the whole configuration of the AV inf or- 
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mation process is described with reference to FIG. 2. 
FIG. 2 is a flowchart schematically showing the respec- 
tive processes constituting the AV information process 
in a module manner and also showing a flow of the in- 
formation and the relation between the respective proc- 
esses. 

[0077] As shown in FIG.2, when the AV information 
process according to the embodiment is carried out, a 
login process LI is firstly carried out. 
[0078] This login process LI is mainly carried out by 
the system manage agent 9 and the user learn agent 4. 
Actually, when a user inputs a voice to the microphone 
1 , a process of identifying a name of the user, a process 
of reading out a usage history for each user associated 
with the identifying process and the like are carried out. 
Then, the result of the identifying process is outputted 
to an input process IP and an accumulated information 
process CK. Incidentally, even while just one user is us- 
ing the AV information processing apparatus A, the login 
process LI is carried out for each voice input from the 
one user. 

[0079] Next, the input process IP is mainly carried out 
by the system manage agent 9, the voice recognize 
agent 2 and the language analyze & compose agent 3. 
Actually, it recognizes the content of a process request 
inputted through the microphone 1 by using the user's 
voice (i.e., a process request including a content of the 
AV information process to be carried out by the AV in- 
formation processing apparatus A), and outputs its re- 
sult to a request analysis process RQ. 
[0080] Then, the request analysis process RQ is the 
process serving as the core in the AV information proc- 
ess of the embodiment. It is mainly carried out by the 
user learn agent 4, the interactive agent 5, the search 
agent 11 , the database agent 12, the system manage 
agent 9 and the AV control agent 10. Actually, it carries 
out the various processes corresponding to the process 
requests inputted by the user, and also instructs a re- 
producing process AP or a recording process AR to car- 
ry out recording the AV information or reproducing the 
AV information necessary for the execution of the proc- 
ess, respectively. 

[0081] Also, the request analysis process RQ carries 
out the AV information process desired by the user in an 
interactive manner with the user by generating a closed 
loop between the request analysis process RQ and the 
input process IP. 

[0082] Moreover, when it becomes necessary to out- 
put a voice to the user in the interaction with the user, 
the request analysis process RQ outputs the information 
indicative of a content to be outputted, to a user re- 
sponse process UR. At this time, if the AV information 
processing apparatus A does not have the information 
corresponding to a process request inputted by the user, 
the request analysis process RQ outputs the information 
indicative of a voice output of some information related 
to the process request to the user response process UR, 
or outputs the information indicative of the absence of 



the pertinent information to the user response process 
UR. 

[0083] Furthermore, when it is judged from the inter- 
action with the user that the user finishes inputting the 

s process request to the AV information processing appa- 
ratus A, the request analysis process RQ outputs end 
information indicative of the end to a logout process LO. 
[0084] On the other hand, the accumulated informa- 
tion process CK receiving the result of the identifying 

10 process outputted by the login process Li is mainly car- 
ried out by the user learn agent 4, the interactive agent 
5, the search agent 11 and the database agent 12. Ac- 
tually, the accumulated information process CK checks 
whether or not there is a process request that is non- 

15 completed in the process request on and after the login 
process LI was performed at the previous time. Further, 
if there is the process request that is non-completed yet 
and the AV information processing apparatus A already 
obtains the AV information under which the pertinent 

20 process request can be completed, the accumulated in- 
formation process CK outputs the information indicating 
that the process request can be completed, to the user 
response process UR. 

[0085] Accordingly, the user response process UR 
25 prepares a response sentence to be used in a response 
to the user corresponding to the information outputted 
by the accumulated information process CK or the in- 
formation indicative of an execution of the voice output 
outputted by the request analysis process RQ, in coin- 
so cidence with a feature of the user, and then outputs the 
response information to an output process OR At this 
time, the user response process UR is mainly carried 
out by the user learn agent 4 and the interactive agent 5. 
[0086] Next, the output process OP is mainly carried 
35 out by the voice synthesize agent 8, the language ana- 
lyze & compose agent 3 and the system manage agent 
9. Concretely, it carries out a process of converting the 
response information outputted by the user response 
process UR, into a voice for an actual output, and then 
40 outputting it through the speaker 7 to the user, and if 
there is information to be outputted as a picture, its con- 
tent is displayed through the system manage agents 9 
and 1 7 on the display 1 8. 

[0087] The reproducing process AP is mainly carried 
45 out by the system manage agent of each device, which 
is connected through the network N to the AV informa- 
tion processing apparatus A and has a function of re- 
producing the AV information. Concretely, the reproduc- 
ing process AP carries out the typical reproducing proc- 
50 ess of the AV information, in accordance with the in- 
struction information from the request analysis process 
RQ, and also feeds back to the request analysis process 
RQ the reproduced AV information, the control informa- 
tion indicative of the end of the reproduction and the like. 
55 [0088] On the other hand, the recording process AR 
is mainly earned out by the system manage agent of 
each device, which is connected through the network N 
to the AV information processing apparatus A and has 
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a function of reproducing the AV information. Concrete- 
ly, the recording process AR carries out the reproducing 
process of the AV information and the typical recording 
process of the reproduced AV information, in accord- 
ance with the instruction information from the request 
analysis process RQ, and also feeds back to the request 
analysis process RQ the control information indicative 
of the end of the recording and the like. 
[0089] The logout process LO is mainly carried out by 
all the system manage agents and the user learn agent 
4. Actually, it carries out a resetting process and an end- 
ing process of the AV information processing apparatus 
A itself, as well as the resetting processes and the end- 
ing processes of the respective connected devices, in 
accordance with the end information from the request 
analysis process RQ. Also, if there is the AV information 
process which has not been completed yet although it 
is desired to be completed in the AV information process 
in this time, the logout process LO carries out a process 
of storing its fact, and then ends the entire AV informa- 
tion process in the embodiment. Incidentally, if a power 
supply of the AV information processing apparatus A it- 
self is not turned off after the end of the logout process 
LO, the operational flow waits for a next start of the login 
process LI. 

[0090] Finally, an information download process DL is 
always carried out independently of the respective proc- 
esses as mentioned above (whether or not the login 
process LI is carried out to thereby start the AV informa- 
tion process), and it is mainly carried out by the user 
learn agent 4 and the download agent 13. Concretely, 
the information download process DL carries out a proc- 
ess of obtaining AV information to complete the non- 
completed AV information process from the external 
network 16 and then recording it to the AV information 
recording portion 14 and also updating the AV informa- 
tion database 15. 

[0091] Next, the detailed portions of the respective 
processes described with reference to FIG.2 will be de- 
scribed below with reference to FIG.3 to FIG.12. 
[0092] At first, the detailed portions of the login proc- 
ess LI is described with reference to FIG.3. FIG.3 is a 
flowchart showing the detailed portions of the login proc- 
ess LI. 

[0093] In FIG. 3, in the login process LI, if it is recog- 
nized that a user starts the operation of the AV informa- 
tion processing apparatus A through the system man- 
age agent 9 for managing a voice signal from the micro- 
phone 1, the user learn agent 4 judges whether or not 
the user is coincident with a user who has used the AV 
information processing apparatus A immediately before 
(Step S1). 

[0094] If the user is coincident with the user immedi- 
ately before i.e., if the user is not changed (Step S1 ; 
NO), it is judged that it is not necessary to obtain or up- 
date a user history in the user learn agent 4. So, the 
content of the input voice is outputted as it is to the sys- 
tem manage agent 9 for carrying out a part of the input 



process IP and the user learn agent 4 for carrying out a 
part of the accumulated information process CK. 
[0095] On the one hand, in the judgment at the step 
S1 , if the user is not coincide with the user immediately 

5 before i.e., if the user is changed (Step S1 ; YES), the 
user learn agent 4 judges whether or not a previous us- 
age history of the recognized user (namely, the usage 
history of the recognized user among the usage histo- 
ries of the AV information processing apparatus A accu- 
mulated for each user) is present in the user learn agent 
4 (Step S2). If it is present (Step S2 ; YES), the corre- 
sponding usage history is read from the user learn agent 
4 and is loaded to the request analysis process RQ 
(Step S3). Also, the content of the input voice is output- 

15 ted to the system manage agent 9 for carrying out a part 
of the input process IP and the user learn agent 4 for 
carrying out a part of the accumulated information proc- 
ess CK. 

[0096] In the judgment at the step S2, if the previous 
20 usage history of the recognized user is not present in 
the user learn agent 4 (Step S2 ; NO), a new usage his- 
tory corresponding to the recognized user is prepared 
in the user learn agent 4 (Step S4). After that, the con- 
tent of the input voice is outputted to the system manage 
25 agent 9 for carrying out a part of the input process IP 
and the user learn agent 4 for carrying out a part of the 
accumulated information process CK.. 
[0097] Incidentally, the above explained login process 
LI is started in accordance with an output of a voice from 
30 the user. Other than it, for example, it may be designed 
such that identification information in an identification 
card of each user is read in by a card reader (not shown) 
to thereby identify each user and then start the login 
process LI. 

35 [0098] Next, the detailed portions of the input process 
IP will be described below with reference to FIG. 4. FIG. 
4 is a flowchart showing the detailed portions of the input 
process IP. 

[0099] In FIG. 4, in the input process IP, the system 

40 manage agent 9 firstly judges whether or not the infor- 
mation outputted by the login process LI is the voice in- 
formation (Step S5). If it is the voice information (Step 
S2 ; YES, and it becomes usually "Yes" in the present 
embodiment), the voice information is recognized by the 

45 voice recognize agent 2 (Step S6). After that, the rec- 
ognized voice information is translated into a middle lan- 
guage by the language analyze & compose agent 3 
(Step S7). Then, the information indicative of the trans- 
lated meaning and content (i.e., the meaning and con- 

50 tent entered as a process request through a voice of the 
user) is outputted to the interactive agent 5 for carrying 
out a part of the request analysis process RQ. 
[0100] On the one hand, in a case that the AV infor- 
mation processing apparatus A jointly has a function of 

55 recognizing, for example, a handwritten character and 
receiving a process request of the handwritten character 
or a function of receiving a process request using a re- 
mote control prepared in advance, if the information out- 
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putted by the login process LI is the character recogni- 
tion result or the input result through the remote control 
in the judgment at the step S5 (Step S5 ; NO), the re- 
spective input contents are directly recognized by the 
system manage agent 9 and the like (Step S8). Then, 5 
the information indicative of the recognized input con- 
tent is outputted to the interactive agent 5 for carrying 
out a part of the request analysis process RQ. 
[01 01] Next, the detailed portions of the request anal- 
ysis process RQ will be described below with reference 10 
to FIG.5. FIG.5 is a flowchart showing the detailed por- 
tions of the request analysis process RQ. 
[01 02] In FIG. 5, at first, the request analysis process 
RQ again checks the usage history corresponding to 
each user stored in the user learn agent 4, in accord- 15 
ance with the voice recognition result inputted to the in- 
teractive agent 5, and then updates its content (e.g., a 
usage frequency of the AV information processing ap- 
paratus A) (Step S9). Then, the interactive agent 5 judg- 
es whether or not a process corresponding to the proc- 20 
ess request of the user included in the input voice rec- 
ognition result can be executed in the AV information 
processing apparatus A (Step S1 0). 
[01 03] if the process corresponding to the process re- 
quest cannot be executed in the AV information 25 
processing apparatus A (Step S10; NO), the search 
agent 1 1 and the database agent 1 2 are used to search 
the AV information database 15 and the external net- 
work 1 6 for the information related with the process re- 
quest (e.g., if the process request is to reproduce a cer- 30 
tain song of a certain singer, such a process is per- 
formed to search for another song of this singer, or 
search for a song of another singer in the genre same 
as this singer, or search for the newest song of this sing- 
er, or the like) (Step S1 1 ). 35 
[0104] Then, the search agent 11 judges whether or 
not any search result can be obtained (Step S12). If the 
search result can be obtained (Step S12 ; YES), the in- 
teractive agent 5 performs a process of synthesizing a 
voice output corresponding to the obtained search result *o 
(Step S1 4). Its result is outputted to the user learn agent 
4 for carrying out a part of the user response process 
UR. 

[0105] On the one hand, in the judgment at the step 
S12, if any search result cannot be obtained (Step S12 ; 45 
NO), the interactive agent 5 generates an exception 
message (actually, a message of "Request cannot be 
accommodated" and the like) (Step S13), and performs 
the process of synthesizing the corresponding voice 
output (Step S1 4). so 
[01 06] On the other hand, in the judgment at the step 
S10, if the process corresponding to the process re- 
quest from the user can be executed in the AV informa- 
tion processing apparatus A (Step S10 ; YES), the in- 
teractive agent 5 judges whether or not the process re- 55 
quest has the content requesting a presentation of the 
information desired by the user (e.g., a search display 
of a group of songs having a certain common feature, 



or a presentation of a song that is being reproduced cur- 
rently and frequently, or the like) (Step S15). 
[0107] Then, if the process request requests the pres- 
entation of the information (Step S15 ; YES), the inter- 
active agent 5 instructs the search agent 11 to search 
for the information corresponding to the process request 
within the AV information database 15 and the external 
network 16. So, the search agent 11 and the database 
agent 1 2 search for the corresponding information (Step 
S16). 

[0108] Then, the search agent 11 judges whether or 
not the corresponding information can be obtained 
(Step S17). If it can be obtained (Step S17 ; YES), the 
interactive agent 5 performs a process of synthesizing 
a voice output corresponding to the obtained informa- 
tion (Step S14). Its result is outputted to the user leam 
agent 4 for carrying out a part of the user response proc- 
ess UR. 

[0109] On the one hand, in the judgment at the step 
S17, if the corresponding information cannot be ob- 
tained (Step S1 7 ; NO), this implies that the information 
representation request at the current time cannot be 
met. Thus, the interactive agent 5 outputs the informa- 
tion indicative of the non-treated process request (i.e., 
the process request for the non-completed process) to 
the user leam agent 4. Also, the user leam agent 4 reg- 
isters its fact in the corresponding usage history (Step 
S29). Moreover, in order to search for the information 
related with the process request, the operational flow 
proceeds to the step S11 . After that, the processes at 
the steps S11 to S14 are earned out. 
[01 10] On the other hand, in the judgment at the step 
S15, if the process request does not request the repre- 
sentation of the information (Step S15 ; NO), the inter- 
active agent 5 judges whether or not the process re- 
quest requests a representation of a list of a song de- 
sired by the user and the like (Step S1 8). 
[0111] If the process request requests the presenta- 
tion of the list (Step S18 ; YES), the interactive agent 5 
outputs an edition condition in the list to be edited and 
the like, to the edit agent 6. Also, the interactive agent 
5 instructs the search agent 11 to search for the infor- 
mation to be the list through the AV information data- 
base 15 and the external network 16. So, the search 
agent 11 and the database agent 12 search for the cor- 
responding information (Step S19). 
[0112] Then, the database agent 12 outputs the ob- 
tained information to the edit agent 6. Also, the edit 
agent 6 carries out an operation for editing the list (Step 
S20). 

[0113] After that, the edit agent 6 judges whether or 
not the necessary list can be edited (Step S21 ). If it can 
be edited (Step S21 ; YES), the interactive agent 5 car- 
ries out a process of displaying the obtained list on the 
display 1 8 (Step S14). Its result is outputted to the user 
leam agent 4 for carrying out a part of the user response 
process UR. 

[0114] On the one hand, in the judgment at the step 
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S21 , if the necessary list cannot be edited since the nec- 
essary information cannot be searched or because of 
other reasons (Step S21 ; NO), this implies that the list 
preparation request at the current time cannot be met. 
So, the operational flow proceeds to the step S29. More- 5 
over, in order to search for the information related with 
the list edition request, the operational flow proceeds to 
the step S1 1 . After that, the processes at the steps S1 1 
to S14 are carried out. 

[0115] On the other hand, in the judgment at the step io 
S1 8, if the process request does not request the repre- 
sentation of the list (Step S18; NO), the interactive 
agent 5 judges whether or not the process request re- 
quests recording or reproducing the AV information 
such as a song desired by the user and the like (actually, 1$ 
for example, recording in MD a song recorded in the AV 
information recording portion 14) (Step S22). 
[01 1 6] If the process request requests recording or re- 
producing the AV information (Step S22 ; YES), the in- 
teractive agent 5 outputs to the system manage agent 20 

9 the instruction of detecting the operation states of the 
respective devices connected to the AV information ac- 
cumulating apparatus S. The system manage agent 9 
checks the operation states (e.g., an on/off state of a 
power supply or an occurrence state of a trouble) of the 25 
respective devices connected through the network N, 
and outputs its result to the interactive agent 5 (Steps 
S23 and S24). 

[0117] If the recording or reproducing process includ- 
ed in the process request cannot be carried out as the 30 
result of the check at the step S24 (Step S24 ; NO), the 
operational flow proceeds to the step S29. Moreover, in 
order to search for the information related with the re- 
cording or reproducing process request, the operational 
flow proceeds to the step S1 1 . After that, the processes 35 
at the steps S11 to S14 are carried out. 
[0118] On the other hand, if the recording or repro- 
ducing process included in the process request can be 
carried out in the check at the step S24 (Step S24 ; 
YES), its fact is indicated to the user. Next, the system 40 
manage agent 9 checks whether or not there is an input 
indicative of an admission of a process start from the 
user through the interactive agent 5 (Step S25). 
[0119] If the process start is admitted (Step S25 ; 
YES), the system manage agent 9 reports the start of 45 
the correspondingly recording or reproducing process, 
to the AV control agent 10. Also, the AV control agent 

10 receiving this report outputs the information with re- 
gard to an indication of an order of songs in the recording 

or reproducing process or an indication of the recording so 
or reproducing manner, to the system manage agent 9. 
Then, the actual recording or reproducing process is 
carried out while the system manage agent 9 manages 
the system manage agent 20, 22, 24, 26 or 28 in each 
device (Step S27) . At this time, when the process at the 55 
step S27 is carried out, the recording or reproducing 
process is carried out while the information is sent to or 
received from the reproducing device for carrying out 



the above-mentioned reproducing process AP (actually, 
the system manage agent of the device having the func- 
tion of reproducing the AV information) or the recording 
device for carrying out the above-mentioned recording 
process AR (actually, the system manage agent of the 
device having the function of recording the AV informa- 
tion). 

[0120] After that, after the completion of the neces- 
sarily recording or reproducing process, the AV control 
agent 10 carries out a message preparation process of 
indicating its completion to the user (Step S28). The 
preparation result is outputted to the interactive agent 
5, and the processes on and after the step S1 4 are car- 
ried out. 

[0121] On the one hand, in the judgment at the step 
S25, if the process start is not confirmed (Step S25 ; 
NO), the AV control agent 10 prepares a confirm mes- 
sage indicative of the confirmation for the process start 
to the user (Step S26). This is outputted through the in- 
teractive agent 5 to the user (Step S14). 
[01 22] On the other hand, in the judgment at the step 
S22, if the process request does not request the record- 
ing or reproducing of the AV information (Step S22 ; 
NO), there may be a possibility that the user requests 
the process request except the presentation of the in- 
formation, or the preparation of the list, or the recording 
or reproducing of the AV information. Forcaution's sake, 
the interactive agent 5 checks whether or not the user 
requests the logout process LO in order to end the us- 
age of the AV information processing apparatus A (Step 
S30). if the user requests the logout process LO (Step 
S30 YES), the end information is outputted to the logout 
process LO, as it is. If the user does not request (Step 
S30 ; NO), in order that the side of the AV information 
processing apparatus A positively checks the content of 
the process desired by the user, the operational flow 
proceeds to the step S11 . Then, it searches for the in- 
formation with regard to the process except the presen- 
tation of the information, or the preparation of the list, or 
the recording or reproducing of the AV information (Step 
S11). After that, the processes at the steps S12 to S14 
are carried out. 

[0123] Next, the detailed portions of the accumulated 
information process CK will be described below with ref- 
erence to FIG.6. FIG.6 is a flowchart showing the de- 
tailed portions of the accumulated information process 
CK. 

[01 24] In FIG. 6, in the accumulated information proc- 
ess CK, at first, in accordance with a voice recognition 
result inputted to the interactive agent 5, the user learn 
agent 4 checks whether or not the AV information data- 
base 15 is updated, through the database agent 12 
(StepS31). 

[01 25] If the AV information database 1 5 is not updat- 
ed (Step S31 ; NO), it is judged that there is no new in- 
formation to be given to the user. Thus, the operational 
flow waits for the update, ff the AV information database 
15 is updated (Step S31 ; YES), the user learn agent 4 
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requests the search agent 11 to again search for the in- 
formation corresponding to the non-completed process 
in the usage history stored in the user learn agent 4, and 
the search agent 11 receiving this request sends a 
search condition, a search parameter or the like, to the 
database agent 1 2 to thereby perform the actual search. 
After that, the presence or absence of the search result 
is checked (Step S32). If a new search result corre- 
sponding to the non-completed process is not obtained 
(Step S32 ; NO), the operational flow waits for the up- 
date of the AV information database 15. On the other 
hand, if the new search result is obtained (Step S32 ; 
YES), the search result is outputted from the user learn 
agent 4 to the interactive agent 5, so that a preparation 
to indicate it to the user is carried out (Step S33). Then, 
its preparation result is fed back to the user learn agent 
4 for carrying out a part of the user response process 
UR. 

[0126] Next, the detailed portions of the user re- 
sponse process UR will be described below with refer- 
ence to FIG.7. FIG.7 is a flowchart showing the detailed 
portions of the user response process UR. 
[0127] In FIG. 7, in the user response process UR, 
when the response information prepared by the request 
analysis process RQ (refer to the step S14) or the 
search result prepared by the accumulated information 
process CK (refer to the step S33) is inputted, the inter- 
active agent 5 firstly inquires of the user learn agent 4 
whether or not a user currently using the AV information 
processing apparatus A performs any specification on 
the output manner of the process result (for example, a 
specification to carry out the output through a voice, or 
a specification to carry out the output through a picture 
display, or the like) (Step S34). If any output manner is 
specified (Step S34 ; YES), the corresponding output 
manner is selected in the interactive agent 5 (Step S35). 
Then, the operational flow proceeds to a step S37. 
[0128] On the other hand, if the output manner is not 
specified in the check at the step S34 (Step S34 ; NO), 
either one of pre-set output manners is selected (Step 
S36). If the output manner is a picture display, the inter- 
active agent 5 inquires of the user learn agent 4 whether 
or not the user specifies a display manner (actually, a 
specification of a screen layout for an output and the 
like) (Step S37). If the display manner is specified (Step 
S37 ; YES), a format is generated so as to indicate the 
process result in accordance with the specified display 
manner (Step S3B). The format result is outputted to the 
system manage agent 9 (in a case of the picture display 
output) or the language analyze & compose agent 3 (in 
a case of the voice output) for carrying out a part of the 
output process OP, as the response information. 
[01 29] On the other hand, if the display manner is not 
specified in the check at the step S37 (Step S37 ; NO), 
a format is generated so as to indicate the process result 
in accordance with either one of pre-set display man- 
ners (Step S39). The format result is outputted to the 
system manage agent 9 or the language analyze & com- 



pose agent 3 for carrying out a part of the output process 
OP, as the response information. 
[0130] Next, the detailed portions of the output proc- 
ess OP will be described below with reference to FIG. 
s 8. FIG.8 is a flowchart showing the detailed portions of 
the output process OP. 

[0131] In FIG. 8, in the output process OP, it is firstly 
judged whether or not the output manner specified in 
the user response process UR is the usage of the voice 
io (StepS40). 

[0132] ff it is judged as the usage of the voice (Step 
S40 ; YES), the language analyze & compose agent 3 
converts the content of the process result constituted by 
the middle language into a response sentence carte- 
ls spending to the situation, and outputs the converted re- 
sult to the voice synthesize agent 8 (Step S41). 
[0133] Then, the voice synthesize agent 8 receiving 
this converted result synthesizes the corresponding 
voice, and outputs it through the speaker 7 to the user 
(Step S42). 

[01 34] On the other hand, in the judgment at the step 
S40, if the output manner specified in the user response 
process UR is judged as the usage of the picture display 
(Step S40 ; NO), the corresponding picture is synthe- 
sized by the system manage agents 9 and 17 and is 
outputted to the user (Step S43). 
{0135] Next, the detailed portions of the reproducing 
process AP will be described below with reference to 
FIG.9. FIG.9 is a flowchart showing the detailed portions 
of the reproducing process AP. 
[0136] In FIG. 9, in the reproducing process AP, the 
system manage agent 9 firstly outputs the request in- 
dicative of the reproduction preparation through the net- 
work N to the system manage agent of the device having 
the reproducing function targeted for the AV information 
reproduction (Step S44). Then, the system manage 
agent 9 checks whether or not the device targeted for 
the reproduction is abnormal with regard to the repro- 
ducing function (Step S45). 

[01 37] if the targeted device is abnormal (Step S46 ; 
YES), the desired reproducing function cannot be exe- 
cuted as it is. Thus, its fact is outputted to the system 
manage agent 9 for carrying out the request analysis 
process RQ (refer to the step S23). 
[0138] On the other hand, if it is not abnormal (Step 
S45 ; NO), the system manage agent of the device tar- 
geted for the reproduction performs the actually repro- 
ducing process (Step S46). 

[0139] Afterthat, it is checked whether or not the con- 
trol information indicative of a reproduction end is out- 
putted from the system manage agent on the device 
side targeted for the reproduction to the system manage 
agent 9 (Step S47). If it is outputted (Step S47; YES), 
the system manage agent 9 outputs its fact to the AV 
control agent 1 0 carrying out the process at the step 
S27. 

[0140] On the other hand, if the reproduction is not 
ended in the judgment at the step S47 (Step S47 ; NO), 
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it is always monitored whether or not the device targeted 
for the reproduction is abnormal (Step S48). If the tar- 
geted device is not abnormal (Step S48 ; NO), the op- 
erational flow returns back to the step S47. In succes- 
sion, it continues the reproducing process. If the target- 5 
ed device is abnormal and the continuation of the repro- 
ducing process cannot be continued (Step S48 ; YES), 
the system manage agent 9 outputs its fact to the AV 
control agent 10 carrying out the process at the step 
S27. 10 
[0141] Next, the detailed portions of the recording 
process AR will be described below with reference to 
FIG.10. FIG.1 0 is a flowchart showing the detailed por- 
tions of the recording process AR. 
[0142] In FIG. 10, in the recording process AR, the 1$ 
system manage agent 9 firstly outputs a request indic- 
ative of a reproducing preparation through the network 
N to a system manage agent of a device having the re- 
producing function targeted for the reproducing process 
associated with the recording of AV information (Step 20 

549) . Then, the system manage agent 9 checks wheth- 
er or not the device targeted for the reproduction is ab- 
normal with regard to the reproducing function (Step 

550) . 

[0143] If the targeted device is abnormal (Step S50 ; 25 
YES), it cannot perform the desirably reproducing and 
recording functions as it is. Thus, its fact is outputted to 
the system manage agent 9 for carrying out the request 
analysis process RQ (refer to the step S23). 
[01 44] On the other hand, if the targeted device is not 30 
abnormal (Step S50 ; NO), the system manage agent 9 
then outputs a request indicative of a recording prepa- 
ration through the network N to a system manage agent 
of a device having the recording function targeted for 
the recording of the AV information (Step S51). Then, 35 
the system manage agent 9 checks whether or not the 
device targeted for the recording is abnormal with re- 
gard to the recording function (Step S52). 
[01 45] If the targeted device is abnormal (Step S52 ; 
YES), it cannot perform the desirably recording function 40 
as it is. Thus, its fact is outputted to the system manage 
agent 9 for carrying out the request analysis process RQ 
(refer to the step S23). 

[01 46] On the other hand, if the targeted device is not 
abnormal (Step S52 ; NO), the system manage agent 45 
of the device targeted for the reproduction then performs 
the actual reproducing process. Also, the system man- 
age agent of the device targeted for the recording per- 
forms the process of recording the reproduced AV infor- 
mation (Step S53). so 
[01 47] After that, it is checked wheth er or not the sys- 
tem manage agent on the device side targeted for the 
reproduction outputs the information indicative of an end 
of the reproduction to the system manage agent 9 (Step 
S54). If this information is outputted (Step S54 ; YES), 55 
the system manage agent 9 outputs its fact to the AV 
control agent 1 0 for carrying out the process at the step 
S27. 



[0148] On the other hand, if the reproduction is not 
still ended in the judgment at the step S54 (Step S54 ; 
NO), it is always monitored whether or not any trouble 
is induced in the device targeted for the reproduction 
and the record (Step S55). If any trouble is not induced 
i.e., if it is not abnormal (Step S55 ; NO), the operational 
flow returns back to the step S54. The reproducing and 
recording processes are continued in succession. If any 
trouble occurrence disables the reproducing and re- 
cording processes to be continued i.e., if it is abnormal 
(Step S55 ; YES), the system manage agent 9 outputs 
its fact to the AV control agent 10 for carrying out the 
process at the step S27. 

[0149] Next, the detailed portions of the logout proc- 
ess LO will be described below with reference to FIG. 
11. FIG. 11 is a flowchart showing the detailed portions 
of the logout process LO. 

[01 50] In FIG. 1 1 . in the logout process LO, if a logout 
request is outputted at the step S30 in the request anal- 
ysis process RQ so that the end information is outputted 
(Step S30 ; YES), the interactive agent 5 firstly inquires 
of the AV control agent 1 0 whether or not the recording 
or reproducing the AV information is being currently per- 
formed (Step S56). If it is being currently performed 
(Step S56 ; YES), all the system manage agents carry 
out an initializing process to carry out the logout, in ac- 
cordance with the request from the interactive agent 5 
i.e., the system reset is performed (Step S57). Then, the 
operational flow proceeds to a step S58. 
[01 51] On the other hand, if the AV information is not 
being recorded or reproduced in the judgment at the 
step S56 (Step S56 ; NO), it is then checked whether or 
not there is a non-completed process request in the in- 
teractive agent 5 (Step S58). If there is the non-complet- 
ed process request since the user gets up from a seat 
during the process or because of other reasons (Step . 
S58 ; YES), the fact that the non-completed process re- 
quest remains is registered in a usage history corre- 
sponding to a current user among the usage histories 
stored in the user learn agent 4 (Step S59). Then, in 
order to wait for a next login, the operational flow pro- 
ceeds to a waiting state. 

[01 52] On the other hand, if there is not the non-com- 
pleted process request in the judgment at the step S58 
(Step S58 ; NO), it is allowable to end the process as it 
is. Thus, in order to wait for a next login, the operational 
flow proceeds to the waiting state as it is. 
[0153] Finally, the detailed portions of the information 
download process DL will be described below with ref- 
erence to FIG.12. FIG. 12 is a flowchart showing the de- 
tailed portions of the information download process DL. 
[0154] The information download process DL is the 
process which is always carried out for each predeter- 
mined temporal interval, irrespectively of the presence 
or absence of the login from the user to the AV informa- 
tion processing apparatus A. 

[0155] That is, in FIG. 12, in the information download 
process DL, at first, the usage histories stored in the us- 
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er learn agent 4 are all read out (Step S60). Then, the 
user learn agent 4 checks whether or not there exists a 
non-completed process request in the usage histories 
(Step S61). 

[0156] If there is not the non-completed process re- 
quest (Step S61 ; NO), in order to repeat the information 
download process DL in the predetermined temporal in- 
terval, the operational flow returns back to the step S60. 
On the other hand, if there is the non-completed process 
request (Step S61 ; YES), the user learn agent 4 then 
requests the download agent 13 to obtain the informa- 
tion necessary for the completion of the non-completed 
process request. Correspondingly to this request, the 
download agent 13 searches through the external net- 
work 16 (Step S62). 

[0157] After that, the download agent 13 checks 
whether or not there is the corresponding information as 
the result of the search (Step S63). If there is not the 
corresponding information (Step S63 ; NO), in order to 
repeat the similar process in the information download 
process DL at a next time, the operational flow returns 
back to the step S60. On the other hand, if the corre- 
sponding information can be searched (Step S63; 
YES), the download agent 13 obtains the information 
(Step S64). Moreover, the contents of the AV informa- 
tion recording portion 14 and the AV information data- 
base 1 5 are updated in accordance with the information 
obtained through the database agent 12 (Step S65). 
Then, one information download process DL is ended. 

EXAMPLE 

[0158] One example of the actual operations of the 
AV information processing apparatus A according to the 
present invention will be exemplified and described be- 
low by collecting the above-mentioned respective proc- 
esses. 

[0159] In the above-mentioned case of the AV infor- 
mation process in the AV information processing appa- 
ratus A, for example, if a user says to the microphone 
1 , "is the newest album of a singer "A" accumulated ? 
Although I want to listen to a song of a singer "B" that I 
have ever heard in a theme song, I do not known its title. 
Is there any information ? p , a reply of To my disappoint- 
ment, the newest version of the signer A cannot be still 
downloaded. The newest version "***" of the singer "B" 
is already prepared. Also, as theme songs of movies in 
the songs of the singer B, there are "1 ;+++", "2///" and 
"3;$$$". Please, tell a desirable number." may be replied 
as the reply from the AV information processing appa- 
ratus A. By the way, the replay can be mainly attained 
by the processes at the steps S11 to S1 7 in the above- 
mentioned processes. 

[01 60] Moreover, for the reply, if the user says to the 
microphone 1 , "What kind of song is there in a song of 
a ballade category of R&B for example, an answer 
of ""NT" of a singer "C" in the ballade category is rec- 
ommended as a new song. By the way, it has been 



known that the newest version of the above-mentioned 
signer "A" is arrived three days after. Please, enjoy it 
three days after." may be answered as the answer from 
the AV information processing apparatus A. Incidentally, 

s the answer can be mainly attained by the information 
download process DL and the processes at the steps 
S15 to S17 in the above-mentioned processes. 
[0161] As described above, according to the opera- 
tion of the AV information processing apparatus A in the 

10 embodiment, since the information process is executed 
if the information process is requested which can be ex- 
ecuted by using the accumulated AV information. On the 
other hand, even if the information process is requested 
which cannot be executed by using the accumulated AV 

*5 information, the corresponding related information is 
searched and outputted. Thus, it is possible for the user 
to obtain some kind of process result when the user re- 
quests the information process to the AV information 
processing apparatus A. Thus, it is possible for the user 

20 who is not accustomed to the treatment of the AV infor- 
mation processing apparatus A to easily treat the AV in- 
formation processing apparatus A without dissatisfac- 
tion. In this way, it is possible for the AV information 
processing apparatus A to execute the necessary infor- 
ms mation process in the user friendly manner. 

[0162] Also, since the plurality of related information 
whose response manners are different from each other 
are provided, it is possible for the AV information 
processing apparatus A to execute the information proc- 

30 ess in the even more friendly manner. 

[0163] Moreover, the related information is outputted 
to the external portion by using at least one of the voice 
and the picture. Thus, the related information can be 
outputted in the form that the use can easily recognize it. 

35 [0164] In case thatthe information process which can- 
not be executable by the AV information processing ap- 
paratus A is requested, such an exception message is 
outputted that "We cannot response for your request" or 
the like. Thus, the user can always obtain some re- 

^o sponse when the user requests the information process 
to the AV information processing apparatus A. In this 
way, it is possible for the AV information processing ap- 
paratus A to execute the information process in the even 
more friendly manner. 

45 [0165] Also, the request of the information process 
from the external portion through the voice by using the 
microphone 1 can be received so that the user can eas- 
ily request the information process. 
[0166] incidentally, in the above described embodi- 

50 ment, the method of using the voice has been mainly 
described as the method for sending and receiving the 
information between the user and the AV information 
processing apparatus A. However, other than it, for ex- 
ample, the present invention can be applied to a case 

55 that information is sent and received by using a charac- 
ter recognition and a picture display or that an informa- 
tion is sent and received by using a remote controller 
and the like, a picture display and a voice output. 
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[0167] Also, the program to carry out the processes 
in the respective agents is stored in a flexible disc serv- 
ing as an information record medium. Then, this is read 
out and carried out by a typical personal computer (it is 
necessary that it has a hard disc serving as the AV in- 
formation recording portion 14 and the AV information 
database 15). Accordingly, the personal computer can 
function as the AV information processing apparatus A. 
Alternatively, such a computer program may be down- 
loaded to the AV processing apparatus A through the 
external network 16. 



Claims 

1 . An AV (Audio Visual) information processing appa- 
ratus (A) characterized in that said apparatus com- 
prises: 

an AV information accumulating device (14) for 
accumulating AV information including at least 
one of audio information, video information and 
data information corresponding to at least one 
of the audio information and the video informa- 
tion; 

a requesting device (7) for requesting an exe- 
cution of an information process, which uses 
the accumulated AV information, from an exter- 
nal portion; 

an executing device (1 0) for executing the re- 
quested information process by using the ac- 
cumulated AV information; 
a related information accumulating device (15) 
for accumulating related information, which is 
related to the accumulated AV information; and 
an outputting device (5) for searching the relat- 
ed information accumulating device for the re- 
lated information corresponding to a non-exe- 
cutable process, which is not executable by 
said executing device, and outputting the 
searched related information to an external por- 
tion when the non-executable process is re- 
quested by said requesting device. 

2. An AV information processing apparatus (A) ac- 
cording to Claim 1 , characterized in that said out- 
putting device (5) searches the related information 
accumulating device (15) for a plurality of the relat- 
ed information, whose response manners to the 
non-executable process are differentfrom each oth- 
er, and outputs the searched plurality of the related 
information to the external portion. 

3. An AV information processing apparatus (A) ac- 
cording to Claim 1 or 2, characterized in that said 
outputting device (5) outputs the searched related 
information corresponding to the non-executable 
process to the external portion by using at least one 



of a voice and a picture. 

4. An AV information processing apparatus (A) ac- 
cording to any one of Claims 1 to 3, characterized 

s in that said apparatus further comprises a sub-out- 
putting device (5) for outputting information indicat- 
ing a fact that no process can be executed by said 
AV information processing apparatus if an impossi- 
ble process, which is impossible to execute even by 

10 using the related information, is requested by said 
requesting device (7). 

5. An AV information processing apparatus (A) ac- 
cording to any one of Claims 1 to 4, characterized 

15 in that said requesting device (7) comprises a voice 
receiving device (7) for receiving a request of the 
information process from the external portion 
through a voice. 

20 6. A computer program of instructions executable by 
a computer to perform steps to execute information 
process requested by a user, in an AV information 
processing apparatus (A) comprising: an AV infor- 
mation accumulating device (14) for accumulating 

25 AV information including at least one of audio infor- 
mation, video information and data information cor- 
responding to at least one of the audio information 
and the video information; and a related information 
accumulating device (15) for accumulating related 

30 inf ormation , which is related to the accumu lated AV 
information, characterized in that said steps com- 
prise: 

a requesting step of requesting an execution of 
35 an information process, which uses the accu- 

mulated AV information, from an external por- 
tion; 

an executing step of executing the requested 
information process by using the accumulated 

40 AV information; and 

an outputting step of searching the related in- 
formation accumulating device for the related 
information corresponding to a non-executable 
process, which is not executable by said exe- 

45 cuting step, and outputting the searched relat- 

ed information to an external portion when the 
non-executable process is requested by said 
requesting step. 

so 7. a computer program according to Claim 6, charac- 
terized in that said outputting step searches the re- 
lated information accumulating device (15) for a plu- 
rality of the related information, whose response 
manners to the non-executable process are differ- 

55 ent from each other, and outputs the searched plu- 
rality of the related information to the external por- 
tion. 
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8. A computer program according to Claim 6 or 7, 
characterized in that said outputting step outputs 
the searched related information corresponding to 
the non-executable process to the external portion 

by using at least one of a voice and a picture. 5 

9. A computer program according to any one of Claims 
6 to 8, characterized in that said steps further com- 
prise a sub-outputting step of outputting information 
indicating a fact that no process can be executed w 
by said AV information processing apparatus (A) if 

an impossible process, which is impossible to exe- 
cute even by using the related information, is re- 
quested by said requesting step. 

15 

1 0. A computer program according to any one of Claims 
6 to 9, characterized in that said requesting step 
comprises a voice receiving step of receiving a re- 
quest of the information process from the external 
portion through a voice. 20 
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